Search for: All records

Ensuring the integrity of petabyte-scale file transfers is essential for the data gathered from scientific instruments. As packet sizes increase, so does the likelihood of errors, resulting in a higher probability of undetected errors in the packet. This paper presents a Multi-Level Error Detection (MLED) framework that leverages in-network resources to reduce undetected error probability (UEP) in file transmission. MLED is based on a configurable recursive architecture that organizes communication in layers at different levels, decoupling network functions such as error detection, routing, addressing, and security. Each layer L_ij at level i implements a policy P_ij that governs its operation, including the error detection mechanism used, specific to the scope of that layer. MLED can be configured to mimic the error detection mechanisms of existing large-scale file transfer protocols. The recursive structure of MLED is analyzed and it shows that adding additional levels of error detection reduces the overall UEP. An adversarial error model is designed to introduce errors into files that evade detection by multiple error detection policies. Through experimentation using the FABRIC testbed the traditional approach, with transport- and data link- layer error detection, results in a corrupt file transfer requiring retransmission of the entire file. Using its recursive structure, an implementation of MLED detects and corrects these adversarial errors at intermediate levels inside the network, avoiding file retransmission under non-zero error rates. MLED therefore achieves a 100% gain in goodput over the traditional approach, reaching a goodput of over 800 Mbps on a single connection with no appreciable increase in delay. 

Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications. In pursuit of low-rank solutions and low complexity algorithms, we consider the Burer–Monteiro factorization approach for solving SDPs. For a large class of SDPs, upon random perturbation of the cost matrix, with high probability, we show that all approximate second-order stationary points are approximate global optima for the penalty formulation of appropriately rank-constrained SDPs, as long as the number of constraints scales sub-quadratically with the desired rank. Our result is based on a simple penalty function formulation of the rank-constrained SDP along with a smoothed analysis to avoid worst-case cost matrices. We particularize our results to two applications, namely, Max-Cut and matrix completion.